Modelling human emotions using immersive virtual reality, physiological signals and behavioural responses

Tesis por compendio[ES] El uso de la realidad virtual (RV) se ha incrementado notablemente en la comunidad científica para la investigación del comportamiento humano. En particular, la RV inmersiva ha crecido debido a la democratización de las gafas de realidad virtual o head mounted displays (HMD), que ofrecen un alto rendimiento con una inversión económica. Uno de los campos que ha emergido con fuerza en la última década es el Affective Computing, que combina psicofisiología, informática, ingeniería biomédica e inteligencia artificial, desarrollando sistemas que puedan reconocer emociones automáticamente. Su progreso es especialmente importante en el campo de la investigación del comportamiento humano, debido al papel fundamental que las emociones juegan en muchos procesos psicológicos como la percepción, la toma de decisiones, la creatividad, la memoria y la interacción social. Muchos estudios se han centrado en intentar obtener una metodología fiable para evocar y automáticamente identificar estados emocionales, usando medidas fisiológicas objetivas y métodos de aprendizaje automático. Sin embargo, la mayoría de los estudios previos utilizan imágenes, audios o vídeos para generar los estados emocionales y, hasta donde llega nuestro conocimiento, ninguno de ellos ha desarrollado un sistema de reconocimiento emocional usando RV inmersiva. Aunque algunos trabajos anteriores sí analizan las respuestas fisiológicas en RV inmersivas, estos no presentan modelos de aprendizaje automático para procesamiento y clasificación automática de bioseñales. Además, un concepto crucial cuando se usa la RV en investigación del comportamiento humano es la validez: la capacidad de evocar respuestas similares en un entorno virtual a las evocadas por el espacio físico. Aunque algunos estudios previos han usado dimensiones psicológicas y cognitivas para comparar respuestas entre entornos reales y virtuales, las investigaciones que analizan respuestas fisiológicas o comportamentales están mucho menos extendidas. Según nuestros conocimientos, este es el primer trabajo que compara entornos físicos con su réplica en RV, empleando respuestas fisiológicas y algoritmos de aprendizaje automático y analizando la capacidad de la RV de transferir y extrapolar las conclusiones obtenidas al entorno real que se está simulando. El objetivo principal de la tesis es validar el uso de la RV inmersiva como una herramienta de estimulación emocional usando respuestas psicofisiológicas y comportamentales en combinación con algoritmos de aprendizaje automático, así como realizar una comparación directa entre un entorno real y virtual. Para ello, se ha desarrollado un protocolo experimental que incluye entornos emocionales 360º, un museo real y una virtualización 3D altamente realista del mismo museo. La tesis presenta novedosas contribuciones del uso de la RV inmersiva en la investigación del comportamiento humano, en particular en lo relativo al estudio de las emociones. Esta ayudará a aplicar metodologías a estímulos más realistas para evaluar entornos y situaciones de la vida diaria, superando las actuales limitaciones de la estimulación emocional que clásicamente ha incluido imágenes, audios o vídeos. Además, en ella se analiza la validez de la RV realizando una comparación directa usando una simulación altamente realista. Creemos que la RV inmersiva va a revolucionar los métodos de estimulación emocional en entornos de laboratorio. Además, su sinergia junto a las medidas fisiológicas y las técnicas de aprendizaje automático, impactarán transversalmente en muchas áreas de investigación como la arquitectura, la salud, la evaluación psicológica, el entrenamiento, la educación, la conducción o el marketing, abriendo un nuevo horizonte de oportunidades para la comunidad científica. La presente tesis espera contribuir a caminar en esa senda.[EN] In recent years the scientific community has significantly increased its use of virtual reality (VR) technologies in human behaviour research. In particular, the use of immersive VR has grown due to the introduction of affordable, high performance head mounted displays (HMDs). Among the fields that has strongly emerged in the last decade is affective computing, which combines psychophysiology, computer science, biomedical engineering and artificial intelligence in the development of systems that can automatically recognize emotions. The progress of affective computing is especially important in human behaviour research due to the central role that emotions play in many background processes, such as perception, decision-making, creativity, memory and social interaction. Several studies have tried to develop a reliable methodology to evoke and automatically identify emotional states using objective physiological measures and machine learning methods. However, the majority of previous studies used images, audio or video to elicit emotional statements; to the best of our knowledge, no previous research has developed an emotion recognition system using immersive VR. Although some previous studies analysed physiological responses in immersive VR, they did not use machine learning techniques for biosignal processing and classification. Moreover, a crucial concept when using VR for human behaviour research is validity: the capacity to evoke a response from the user in a simulated environment similar to the response that might be evoked in a physical environment. Although some previous studies have used psychological and cognitive dimensions to compare responses in real and virtual environments, few have extended this research to analyse physiological or behavioural responses. Moreover, to our knowledge, this is the first study to compare VR scenarios with their real-world equivalents using physiological measures coupled with machine learning algorithms, and to analyse the ability of VR to transfer and extrapolate insights obtained from VR environments to real environments. The main objective of this thesis is, using psycho-physiological and behavioural responses in combination with machine learning methods, and by performing a direct comparison between a real and virtual environment, to validate immersive VR as an emotion elicitation tool. To do so we develop an experimental protocol involving emotional 360º environments, an art exhibition in a real museum, and a highly-realistic 3D virtualization of the same art exhibition. This thesis provides novel contributions to the use of immersive VR in human behaviour research, particularly in relation to emotions. VR can help in the application of methodologies designed to present more realistic stimuli in the assessment of daily-life environments and situations, thus overcoming the current limitations of affective elicitation, which classically uses images, audio and video. Moreover, it analyses the validity of VR by performing a direct comparison using highly-realistic simulation. We believe that immersive VR will revolutionize laboratory-based emotion elicitation methods. Moreover, its synergy with physiological measurement and machine learning techniques will impact transversely in many other research areas, such as architecture, health, assessment, training, education, driving and marketing, and thus open new opportunities for the scientific community. The present dissertation aims to contribute to this progress.[CA] L'ús de la realitat virtual (RV) s'ha incrementat notablement en la comunitat científica per a la recerca del comportament humà. En particular, la RV immersiva ha crescut a causa de la democratització de les ulleres de realitat virtual o head mounted displays (HMD), que ofereixen un alt rendiment amb una reduïda inversió econòmica. Un dels camps que ha emergit amb força en l'última dècada és el Affective Computing, que combina psicofisiologia, informàtica, enginyeria biomèdica i intel·ligència artificial, desenvolupant sistemes que puguen reconéixer emocions automàticament. El seu progrés és especialment important en el camp de la recerca del comportament humà, a causa del paper fonamental que les emocions juguen en molts processos psicològics com la percepció, la presa de decisions, la creativitat, la memòria i la interacció social. Molts estudis s'han centrat en intentar obtenir una metodologia fiable per a evocar i automàticament identificar estats emocionals, utilitzant mesures fisiològiques objectives i mètodes d'aprenentatge automàtic. No obstant això, la major part dels estudis previs utilitzen imatges, àudios o vídeos per a generar els estats emocionals i, fins on arriba el nostre coneixement, cap d'ells ha desenvolupat un sistema de reconeixement emocional mitjançant l'ús de la RV immersiva. Encara que alguns treballs anteriors sí que analitzen les respostes fisiològiques en RV immersives, aquests no presenten models d'aprenentatge automàtic per a processament i classificació automàtica de biosenyals. A més, un concepte crucial quan s'utilitza la RV en la recerca del comportament humà és la validesa: la capacitat d'evocar respostes similars en un entorn virtual a les evocades per l'espai físic. Encara que alguns estudis previs han utilitzat dimensions psicològiques i cognitives per a comparar respostes entre entorns reals i virtuals, les recerques que analitzen respostes fisiològiques o comportamentals estan molt menys esteses. Segons els nostres coneixements, aquest és el primer treball que compara entorns físics amb la seua rèplica en RV, emprant respostes fisiològiques i algorismes d'aprenentatge automàtic i analitzant la capacitat de la RV de transferir i extrapolar les conclusions obtingudes a l'entorn real que s'està simulant. L'objectiu principal de la tesi és validar l'ús de la RV immersiva com una eina d'estimulació emocional usant respostes psicofisiològiques i comportamentals en combinació amb algorismes d'aprenentatge automàtic, així com realitzar una comparació directa entre un entorn real i virtual. Per a això, s'ha desenvolupat un protocol experimental que inclou entorns emocionals 360º, un museu real i una virtualització 3D altament realista del mateix museu. La tesi presenta noves contribucions de l'ús de la RV immersiva en la recerca del comportament humà, en particular quant a l'estudi de les emocions. Aquesta ajudarà a aplicar metodologies a estímuls més realistes per a avaluar entorns i situacions de la vida diària, superant les actuals limitacions de l'estimulació emocional que clàssicament ha inclòs imatges, àudios o vídeos. A més, en ella s'analitza la validesa de la RV realitzant una comparació directa usant una simulació altament realista. Creiem que la RV immersiva revolucionarà els mètodes d'estimulació emocional en entorns de laboratori. A més, la seua sinergia al costat de les mesures fisiològiques i les tècniques d'aprenentatge automàtic, impactaran transversalment en moltes àrees de recerca com l'arquitectura, la salut, l'avaluació psicològica, l'entrenament, l'educació, la conducció o el màrqueting, obrint un nou horitzó d'oportunitats per a la comunitat científica. La present tesi espera contribuir a caminar en aquesta senda.Marín Morales, J. (2020). Modelling human emotions using immersive virtual reality, physiological signals and behavioural responses [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/148717TESISCompendi

Product Validation in Creative Processes: A Gender Perspective in Industrial Design Projects

[EN] Design education and practice are continuously evolving. Educational institutions must include intellectual complexities and new curriculum to support good design education. The design education future emerges multidisciplinary knowledge, teaching innovation and employment necessities. This paper describes a methodology centered in product validation with industrial design students. Focusing on discovering the student experience during the project execution, in addition to observing closely the female design student's perception on the methodology and process developed. The academic project was the design of a novel tool board. The students developed the proposed project in a period of eight weeks. Sixteen students participated as a sample of this research. The methodology consisted of eight phases that spanned from project brief to project conclusion, introducing two phases focused on validation exercises for the elements created to reach the solution of the tool board. During the end of the two evaluation phases, two surveys were applied asking for information on his previous experience during his design education and three elements that assessment the design methodology implementation: utility, novelty, and relevance. Using multiple choice and Likert scale answers the students answered the surveys. The survey's findings revealed relevant information on the project implementation focused on evaluation phases during the product design. The results revealed how students reflected on their previous experience developing projects, and how the design tool board integrate important phases like validation. Also, the students evaluated with a positive value the utility, novelty, and relevance of the developed project. However, the most important finding was the female perception comparing male students. The female assessment of novelty and relevance increased during project implementation, highlighting novelty as a perceived element to a greater range than men. This research results allowed us to discover more information about female students experience with creative and validation processes.The authors would also like to acknowledge the financial support provided by the NOVUS grant ID: N20-158-41 (Validación científica como herramienta educativa en proyectos de carácter creativo), as well as the support of the Writing Lab and TecLabs at Tecnologico de Monterrey, Mexico, throughout the production of this work.Rojas, J.; Higuera-Trujillo, JL.; Muniz, G.; Marín-Morales, J. (2021). Product Validation in Creative Processes: A Gender Perspective in Industrial Design Projects. IEEE. 760-765. https://doi.org/10.1109/EDUCON46332.2021.9453948S76076

Development and Calibration of an Eye-Tracking Fixation Identification Algorithm for Immersive Virtual Reality

[EN] Fixation identification is an essential task in the extraction of relevant information from gaze patterns; various algorithms are used in the identification process. However, the thresholds used in the algorithms greatly affect their sensitivity. Moreover, the application of these algorithm to eye-tracking technologies integrated into head-mounted displays, where the subject's head position is unrestricted, is still an open issue. Therefore, the adaptation of eye-tracking algorithms and their thresholds to immersive virtual reality frameworks needs to be validated. This study presents the development of a dispersion-threshold identification algorithm applied to data obtained from an eye-tracking system integrated into a head-mounted display. Rules-based criteria are proposed to calibrate the thresholds of the algorithm through different features, such as number of fixations and the percentage of points which belong to a fixation. The results show that distance-dispersion thresholds between 1-1.6 degrees and time windows between0.25-0.4s are the acceptable range parameters, with 1 degrees and0.25s being the optimum. The work presents a calibrated algorithm to be applied in future experiments with eye-tracking integrated into head-mounted displays and guidelines for calibrating fixation identification algorithmsWe thank Pepe Roda Belles for the development of the virtual reality environment and the integration of the HMD with Unity platform. We also thank Masoud Moghaddasi for useful discussions and recommendations.Llanes-Jurado, J.; Marín-Morales, J.; Guixeres Provinciale, J.; Alcañiz Raya, ML. (2020). Development and Calibration of an Eye-Tracking Fixation Identification Algorithm for Immersive Virtual Reality. Sensors. 20(17):1-15. https://doi.org/10.3390/s20174956S1152017Cipresso, P., Giglioli, I. A. C., Raya, M. A., & Riva, G. (2018). The Past, Present, and Future of Virtual and Augmented Reality Research: A Network and Cluster Analysis of the Literature. Frontiers in Psychology, 9. doi:10.3389/fpsyg.2018.02086Chicchi Giglioli, I. A., Pravettoni, G., Sutil Martín, D. L., Parra, E., & Raya, M. A. (2017). A Novel Integrating Virtual Reality Approach for the Assessment of the Attachment Behavioral System. Frontiers in Psychology, 8. doi:10.3389/fpsyg.2017.00959Marín-Morales, J., Higuera-Trujillo, J. L., De-Juan-Ripoll, C., Llinares, C., Guixeres, J., Iñarra, S., & Alcañiz, M. (2019). Navigation Comparison between a Real and a Virtual Museum: Time-dependent Differences using a Head Mounted Display. Interacting with Computers, 31(2), 208-220. doi:10.1093/iwc/iwz018Kober, S. E., Kurzmann, J., & Neuper, C. (2012). Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study. International Journal of Psychophysiology, 83(3), 365-374. doi:10.1016/j.ijpsycho.2011.12.003Borrego, A., Latorre, J., Llorens, R., Alcañiz, M., & Noé, E. (2016). Feasibility of a walking virtual reality system for rehabilitation: objective and subjective parameters. Journal of NeuroEngineering and Rehabilitation, 13(1). doi:10.1186/s12984-016-0174-1Clemente, M., Rodríguez, A., Rey, B., & Alcañiz, M. (2014). Assessment of the influence of navigation control and screen size on the sense of presence in virtual reality using EEG. Expert Systems with Applications, 41(4), 1584-1592. doi:10.1016/j.eswa.2013.08.055Borrego, A., Latorre, J., Alcañiz, M., & Llorens, R. (2018). Comparison of Oculus Rift and HTC Vive: Feasibility for Virtual Reality-Based Exploration, Navigation, Exergaming, and Rehabilitation. Games for Health Journal, 7(3), 151-156. doi:10.1089/g4h.2017.0114Jensen, L., & Konradsen, F. (2017). A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies, 23(4), 1515-1529. doi:10.1007/s10639-017-9676-0Jost, T. A., Drewelow, G., Koziol, S., & Rylander, J. (2019). A quantitative method for evaluation of 6 degree of freedom virtual reality systems. Journal of Biomechanics, 97, 109379. doi:10.1016/j.jbiomech.2019.109379Chandrasekera, T., Fernando, K., & Puig, L. (2019). Effect of Degrees of Freedom on the Sense of Presence Generated by Virtual Reality (VR) Head-Mounted Display Systems: A Case Study on the Use of VR in Early Design Studios. Journal of Educational Technology Systems, 47(4), 513-522. doi:10.1177/0047239518824862Bălan, O., Moise, G., Moldoveanu, A., Leordeanu, M., & Moldoveanu, F. (2020). An Investigation of Various Machine and Deep Learning Techniques Applied in Automatic Fear Level Detection and Acrophobia Virtual Therapy. Sensors, 20(2), 496. doi:10.3390/s20020496Armstrong, T., & Olatunji, B. O. (2012). Eye tracking of attention in the affective disorders: A meta-analytic review and synthesis. Clinical Psychology Review, 32(8), 704-723. doi:10.1016/j.cpr.2012.09.004Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372-422. doi:10.1037/0033-2909.124.3.372Irwin, D. E. (1992). Memory for position and identity across eye movements. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(2), 307-317. doi:10.1037/0278-7393.18.2.307Tanriverdi, V., & Jacob, R. J. K. (2000). Interacting with eye movements in virtual environments. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI ’00. doi:10.1145/332040.332443Skulmowski, A., Bunge, A., Kaspar, K., & Pipa, G. (2014). Forced-choice decision-making in modified trolley dilemma situations: a virtual reality and eye tracking study. Frontiers in Behavioral Neuroscience, 8. doi:10.3389/fnbeh.2014.00426Juvrud, J., Gredebäck, G., Åhs, F., Lerin, N., Nyström, P., Kastrati, G., & Rosén, J. (2018). The Immersive Virtual Reality Lab: Possibilities for Remote Experimental Manipulations of Autonomic Activity on a Large Scale. Frontiers in Neuroscience, 12. doi:10.3389/fnins.2018.00305Hessels, R. S., Niehorster, D. C., Nyström, M., Andersson, R., & Hooge, I. T. C. (2018). Is the eye-movement field confused about fixations and saccades? A survey among 124 researchers. Royal Society Open Science, 5(8), 180502. doi:10.1098/rsos.180502Diaz, G., Cooper, J., Kit, D., & Hayhoe, M. (2013). Real-time recording and classification of eye movements in an immersive virtual environment. Journal of Vision, 13(12), 5-5. doi:10.1167/13.12.5Duchowski, A. T., Medlin, E., Gramopadhye, A., Melloy, B., & Nair, S. (2001). Binocular eye tracking in VR for visual inspection training. Proceedings of the ACM symposium on Virtual reality software and technology - VRST ’01. doi:10.1145/505008.505010Lim, J. Z., Mountstephens, J., & Teo, J. (2020). Emotion Recognition Using Eye-Tracking: Taxonomy, Review and Current Challenges. Sensors, 20(8), 2384. doi:10.3390/s20082384Manor, B. R., & Gordon, E. (2003). Defining the temporal threshold for ocular fixation in free-viewing visuocognitive tasks. Journal of Neuroscience Methods, 128(1-2), 85-93. doi:10.1016/s0165-0270(03)00151-1Salvucci, D. D., & Goldberg, J. H. (2000). Identifying fixations and saccades in eye-tracking protocols. Proceedings of the symposium on Eye tracking research & applications - ETRA ’00. doi:10.1145/355017.355028Duchowski, A., Medlin, E., Cournia, N., Murphy, H., Gramopadhye, A., Nair, S., … Melloy, B. (2002). 3-D eye movement analysis. Behavior Research Methods, Instruments, & Computers, 34(4), 573-591. doi:10.3758/bf03195486Bobic, V., & Graovac, S. (2016). Development, implementation and evaluation of new eye tracking methodology. 2016 24th Telecommunications Forum (TELFOR). doi:10.1109/telfor.2016.7818800Sidenmark, L., & Lundström, A. (2019). Gaze behaviour on interacted objects during hand interaction in virtual reality for eye tracking calibration. Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications. doi:10.1145/3314111.3319815Alghamdi, N., & Alhalabi, W. (2019). Fixation Detection with Ray-casting in Immersive Virtual Reality. International Journal of Advanced Computer Science and Applications, 10(7). doi:10.14569/ijacsa.2019.0100710Blignaut, P. (2009). Fixation identification: The optimum threshold for a dispersion algorithm. Attention, Perception, & Psychophysics, 71(4), 881-895. doi:10.3758/app.71.4.881Shic, F., Scassellati, B., & Chawarska, K. (2008). The incomplete fixation measure. Proceedings of the 2008 symposium on Eye tracking research & applications - ETRA ’08. doi:10.1145/1344471.1344500Vive Pro Eyehttps://www.vive.com/us

Bases metodológicas para una nueva plataforma de medida del comportamiento humano en entornos virtuales

[ES] Para evaluar la funcionalidad y el rendimiento de un espacio se analiza el comportamiento de sus usuarios. Este se ha medido tradicionalmente a partir de encuestas y observación, con las limitaciones de tratarse de valoraciones subjetivas, influenciadas por el entrevistador y/o observador, y, en el caso de la observación, evaluar el espacio a posteriori, una vez ejecutado el proyecto. Hoy en día, la realidad virtual solventa estos problemas, al ser capaz de representar escenarios de forma realista, inmersiva e interactiva, permitiendo analizar con un bajo coste el comportamiento de los usuarios antes de que se ejecuten los proyectos, en un entorno controlado. El presente artículo presenta las bases metodológicas para una nueva plataforma de medida del comportamiento humano en entornos virtuales, que ayudará en la toma de decisiones a través de la pre-evaluación de los espacios antes de ser ejecutados. Se define una metodología aplicable con la tecnología actual, a partir de la cual se obtendrán métricas con las que optimizar la funcionalidad y el rendimiento de espacios de futura construcción o remodelación de los ya existentes. La herramienta es transversal ya que puede aplicarse a cualquier proyecto que tenga como elemento fundamental el tránsito de personas, ya sean espacios comerciales, culturales, dotacionales o de ocio, y se presentan diferentes ejemplos de aplicación práctica.[EN] Human behavior is analyzed to evaluate the functionality and efficiency of a public space. It was classically measured from surveys and observation, however, those measurements have some limitations. Firstly, they are subjective valuations and are influenced by the interviewer and/or the observer. In addition, the observation oblige us to make that evaluation subsequently, when the project has been executed. Nowadays, virtual reality resolves those problems as a result of its capacity to represent scenarios on a realistic, immersive and interactive way. It allows to analyze human behaviour before the execution of projects at a low cost and controlled way. This article presents the methodological bases for a new platform for measuring human behaviour in virtual environments. It will assist in the decision-making process through the pre-evaluation of different spaces before being executed. An applicable methodology were explained from which metrics are created and it allows to optimize functionality and efficiency of a new construction or remodeling. This is a cross-wise platform and can be applied to any project where the human transit is a central element: commercial, cultural, dotacional or leisure spaces. Different applied examples in study were presented.The present research has been financed by the Ministry of Economy and Competitiveness. Spain (project TIN2013-45736-R).Marín-Morales, J.; Torrecilla-Moreno, C.; Guixeres Provinciale, J.; Llinares Millán, MDC. (2017). Methodological bases for a new platform for the measurement of human behaviour in virtual environments. DYNA: Ingeniería e Industria. 92(1):34-38. https://doi.org/10.6036/7963S343892

Emotion Recognition in Immersive Virtual Reality: From Statistics to Affective Computing

[EN] Emotions play a critical role in our daily lives, so the understanding and recognition of emotional responses is crucial for human research. Affective computing research has mostly used non-immersive two-dimensional (2D) images or videos to elicit emotional states. However, immersive virtual reality, which allows researchers to simulate environments in controlled laboratory conditions with high levels of sense of presence and interactivity, is becoming more popular in emotion research. Moreover, its synergy with implicit measurements and machine-learning techniques has the potential to impact transversely in many research areas, opening new opportunities for the scientific community. This paper presents a systematic review of the emotion recognition research undertaken with physiological and behavioural measures using head-mounted displays as elicitation devices. The results highlight the evolution of the field, give a clear perspective using aggregated analysis, reveal the current open issues and provide guidelines for future research.This research was funded by European Commission, grant number H2020-825585 HELIOS.Marín-Morales, J.; Llinares Millán, MDC.; Guixeres Provinciale, J.; Alcañiz Raya, ML. (2020). Emotion Recognition in Immersive Virtual Reality: From Statistics to Affective Computing. Sensors. 20(18):1-26. https://doi.org/10.3390/s20185163S126201

The background music-content congruence of TV advertisements: A neurophysiological study

[EN] Music affects viewers¿ responses to advertisements. In this study we present the findings of an experiment that investigates the emotional and cognitive reactions of subjects¿ brains during exposure to television advertisements with music congruent, and incongruent, with the advertisement content. We analyze the electroencephalography signals and eye-tracking behaviors of a group of 90 women watching six TV advertisements. The study's findings suggested that incongruent music generates higher levels of attention and advertisement recall. On the other hand, frontal asymmetry measured through electroencephalography was shown to be higher with congruent music. Similarly, cognitive workload was higher when the music was incongruent with the advertisement content. No significant differences were found in terms of advertisement likeability based on incongruent versus congruent music. The results demonstrated the validity of neurophysiological techniques for assessing the effects of levels of music congruence in advertisements.Ausin, JM.; Bigné, E.; Marín-Morales, J.; Guixeres Provinciale, J.; Alcañiz Raya, ML. (2021). The background music-content congruence of TV advertisements: A neurophysiological study. European Research on Management and Business Economics. 27(2):1-14. https://doi.org/10.1016/j.iedeen.2021.100154S11427

Alzheimer Disease Classification through ASR-based Transcriptions: Exploring the Impact of Punctuation and Pauses

Alzheimer's Disease (AD) is the world's leading neurodegenerative disease, which often results in communication difficulties. Analysing speech can serve as a diagnostic tool for identifying the condition. The recent ADReSS challenge provided a dataset for AD classification and highlighted the utility of manual transcriptions. In this study, we used the new state-of-the-art Automatic Speech Recognition (ASR) model Whisper to obtain the transcriptions, which also include automatic punctuation. The classification models achieved test accuracy scores of 0.854 and 0.833 combining the pretrained FastText word embeddings and recurrent neural networks on manual and ASR transcripts respectively. Additionally, we explored the influence of including pause information and punctuation in the transcriptions. We found that punctuation only yielded minor improvements in some cases, whereas pause encoding aided AD classification for both manual and ASR transcriptions across all approaches investigated

Validación de un simulador de conducción de bajo coste para el diseño de carreteras convencionales

[ES] La cantidad de estudios de seguridad vial basados en simuladores de conducción está en continuo crecimiento. En este sentido, la Universitat Politècnica de València (UPV) ha desarrollado un simulador de conducción de bajo coste: SE2RCO (Simulador para la Evaluación, Entrenamiento y Rehabilitación de Conductores). El principal objetivo de la investigación es la validación de este simulador, con el fin de desarrollar estudios relacionados con la seguridad vial y el diseño geométrico de carreteras incorporando el factor humano. Dicha validación se ha realizado a partir de la observación en campo de los perfiles continuos de velocidad desarrollados por 28 voluntarios conduciendo su propio vehículo en un tramo de carretera convencional de 30 km. Los mismos voluntarios condujeron posteriormente en el simulador de conducción ese mismo tramo de carretera reconstruido en un entorno virtual. Un total de 79 curvas y 52 rectas fueron objeto de análisis. La comparación entre las velocidades desarrolladas en la realidad y las observadas durante la simulación permitieron llevar a cabo la validez objetiva del simulador de conducción. Los resultados mostraron que la velocidad media en el simulador y en la realidad era similar cuando la velocidad simulada era inferior a 87.3 km/h. En caso de ser superior, la velocidad media en la realidad era menor que en el simulador. En cuanto a la velocidad de operación, se observó que la velocidad real era aproximadamente 5 km/h menor que la simulada. Finalmente, estos resultados estuvieron apoyados por la percepción de los conductores, ya que la mayoría de ellos evaluaron la calidad del entorno simulado y el grado de similitud entre la tarea de conducción real y simulada como medio o alto, consiguiendo de esta manera la validez subjetiva del simulador de conducción.Los autores quisieran agradecer a la Universitat Politècnica de València (UPV), que financió el proyecto de investigación “CONSIM - Desarrollo de un Modelo para la Evaluación de la Consistencia del Diseño Geométrico de Carreteras Convencionales mediante Simuladores de Conducción” (PAID 05-2012). Asimismo, agradecer también al Ministerio de Economía y Competitividad y al Fondo Social Europeo, que financiaron el proyecto de investigación “CASEFU - Estudio experimental de la funcionalidad y seguridad de las carreteras convencionales” (TRA2013-42578-P), del cual forma parte este estudio.Llopis-Castelló, D.; Camacho Torregrosa, FJ.; Pérez Zuriaga, AM.; Marín-Morales, J.; García García, A.; Dols Ruiz, JF. (2016). Validación de un simulador de conducción de bajo coste para el diseño de carreteras convencionales. En XII Congreso de ingeniería del transporte. 7, 8 y 9 de Junio, Valencia (España). Editorial Universitat Politècnica de València. 1866-1879. https://doi.org/10.4995/CIT2016.2016.3444OCS1866187

Do You See What I See? Effectiveness of 360-Degree vs. 2D Video Ads Using a Neuroscience Approach

[EN] This study compares cognitive and emotional responses to 360-degree vs. static (2D) videos in terms of visual attention, brand recognition, engagement of the prefrontal cortex, and emotions. Hypotheses are proposed based on the interactivity literature, cognitive overload, advertising response model and motivation, opportunity, and ability theoretical frameworks, and tested using neurophysiological tools: electroencephalography, eye-tracking, electrodermal activity, and facial coding. The results revealed that gaze view depends on ad content, visual attention paid being lower in 360-degree FMCG ads than in 2D ads. Brand logo recognition is lower in 360-degree ads than in 2D video ads. Overall, 360-degree ads for durable products increase positive emotions, which carries the risk of non-exposure to some of the ad content. In testing four ads for durable goods and fast-moving consumer goods (FMCG) this research explains the mechanism through which 360-degree video ads outperform standard versionsThis work was supported by the Generalitat Valenciana funded project "Rebrand", grant number PROMETEU/2019/105, and by the European Regional Development Fund program of the Valencian Community 2014-2020 project "Interfaces de realidad mixta aplicada a salud y toma de decisiones", grant number IDIFEDER/2018/029.Ausin-Azofra, JM.; Bigné, E.; Ruiz, C.; Marín-Morales, J.; Guixeres Provinciale, J.; Alcañiz Raya, ML. (2021). Do You See What I See? Effectiveness of 360-Degree vs. 2D Video Ads Using a Neuroscience Approach. Frontiers in Psychology. 12:1-14. https://doi.org/10.3389/fpsyg.2021.612717S1141

An Immersive Virtual Reality Game for Predicting Risk Taking through the Use of Implicit Measures

[EN] The tool presented in this article can be applied as an ecological measure for evaluating decision-making processes in risky situations. It can be used in different contexts from both Occupational Safety and Health practices and for research purposes. Risk taking (RT) measurement constitutes a challenge for researchers and practitioners and has been addressed from different perspectives. Personality traits and temperamental aspects such as sensation seeking and impulsivity influence the individual's approach to RT, prompting risk-seeking or risk-aversion behaviors. Virtual reality has emerged as a suitable tool for RT measurement, since it enables the exposure of a person to realistic risks, allowing embodied interactions, the application of stealth assessment techniques and physiological real-time measurement. In this article, we present the assessment on decision making in risk environments (AEMIN) tool, as an enhanced version of the spheres and shield maze task, a previous tool developed by the authors. The main aim of this article is to study whether it is possible is to discriminate participants with high versus low scores in the measures of personality, sensation seeking and impulsivity, through their behaviors and physiological responses during playing AEMIN. Applying machine learning methods to the dataset we explored: (a) if through these data it is possible to discriminate between the two populations in each variable; and (b) which parameters better discriminate between the two populations in each variable. The results support the use of AEMIN as an ecological assessment tool to measure RT, since it brings to light behaviors that allow to classify the subjects into high/low risk-related psychological constructs. Regarding physiological measures, galvanic skin response seems to be less salient in prediction models.This research was funded by the Spanish Ministry of Economy and Competitiveness funded project "Assessment and Training on Decision Making in Risk Environments", grant number RTC-2017-6523-6, by the Gerenaliat Valenciana funded project "Rebrand", grant number PROMETEU/2019/105, and by the European Union ERDF (European Regional Development Fund) program of the Valencian Community 2014-2020 funded project "Interfaces de realidad mixta aplicada a salud y toma de decisiones", grant number IDIFEDER/2018/029.Juan-Ripoll, CD.; Llanes-Jurado, J.; Chicchi-Giglioli, IA.; Marín-Morales, J.; Alcañiz Raya, ML. (2021). An Immersive Virtual Reality Game for Predicting Risk Taking through the Use of Implicit Measures. Applied Sciences. 11(2):1-21. https://doi.org/10.3390/app11020825S12111